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TIME-DEPENDENT THREE-DIMENSIONAL MUSCULO- 
SKELETAL MODELING BASED ON DYNAMIC SURFACE . 
MEASUREMENTS OF BODIES 

Technical field off ttw invention 
5 Th© Invention rotates* to a system and method for time-dependent three- 

dimensional measurement arid functional analysis of bodies, in particular the 
invention relates to a system and method for obtaining a time-dependent and 
three-dimensional muscaulCHskBletal system for bodies or body parts. The 
invention furthermore relates to a method for quickly and accurately detecting 
10 landmarks on three-dimensional and time-dependent images of body parts and 
to a method for constructing a feasible or approximate rnuscuio-sketefc^ modal 
based on three-dimensione* and time-dependent Images of body parts. 

Background of the invention 

The amount of people suffering from; different muscuio-skeietai 

15 complaints, such as back pain or knee problems, is iarye. As a consequence, 
orthopaedic physicians and physiotherapists are required to anaSyse a variety 
of movements 'of the body to diagnose pathological or abnormal changes. Up 
to now, time-dependent three-dimensional recordings of movement under 
functional conditions are not possible with conventional techniques. As an 

20 ersatz, one or more ihree-dhnensionai scans (ctg. RX) of selected body pails 
{e.g/ knees or feel;} are made under given conditions (e.g. bending/extension 
of the limbs), or video-based kinemetographrc methods (e,g. marker tracing) 
are used to calculate kinematics) parameters (e.g. for gail analysis). An 
overview of the history and technical constraints and different available 

25 systems is given by D.H. Sutherland in "The evolution of* clinical qb\1 analysis.. 
Part II Kinematics", Gait and Posture (2002), 16 t 159-179, 

US Patent application US 2002/0009222 A1 describes a method far 
determining kinetic and kinematic information for a 3D Image of a human body. 
It is based on an input device for images., a transformation system to obtain 3D 

30 .information and a systam for obtaining kinematic and fanatic infafmatfon. The 
image input is based on information of markers placad on the object. This 
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information Is transfomned into & 3D imag© based on an anatomical coordinate 
system of the body segment. Finally the positions and orientations are 
computed in global space for dynamical images. US 2002/0009222 does not 
provide a non-contact technique for obtaining information and furthermore 
5 doss not use muscular modeling. The application of landmarks to the body Is 
time consuming and well tolerated by pattents. 

US Patent application US? 6189817 B1 describes a system and method 
f©r4P kinematic reconstruction and visualization of body tissue. The method \& 
based on segmenting a 3D 'jroag© t following the motion of the different 
10 segments, e.g. based on finite ©lenient models and using the detartee! four- 
dimensional representation of bone, muscles, skin and other tissue as a digital 
clone to study the motion an biomechanics! properties. The document 
describes the construction of a model based on expensive techniques arid 
involving potentially harmful techniques which require careful management. 
15 In "4B analysis of muscular dynamics using flexible 3D muscle model s" a 

Internationa! Conference on Artificial Reality and Telexistenoe '99, SuziKi et aL 
describe a method for octnsuucmg a 4D musculo-skeleta} model. The method 
is based on fitting a muscular and skeletal mode! to MRS results and measuring 
movement by a video camera and a set of sensors, 
20 The above-mentioned documents describe the construction of a 

biomechanics! mode! bs&sed on expensive techniques and potentially harmful 
medical techniques which require careful control to maintain proper patient 
care. Furthermore, the methods described in the aboye mentioned documents 
have the disadvantage that the computing power needed for time-dependent 
25 detection of landmarks on time-dependent images of body parts can be high. 
The above mentioned documents furthermore have the disadvantage that a 
musculoskeletal model only can be obtained based on images of the interior 
of the body parts. Furthermore, She above mentioned documents have the 
disadvantage that the mu sou \ o-ekaletal mode! obtained can lead to bio- 
30 mechanical inconsistent features and that the computing power for obtaining 
the musculoskeletal mode! is large. Dub to the large computing power 
needed, the systems and methods for obtaining a musculoskeletal model are 
tedious and labouNntensive. . . 
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Summary of the invBrtten 

It is an object of the present Invention to provide a method and system 
for efficiently obtaining m dynamical 3 -dimensional muscuio-^skatetal modal for 
5 ' parts of a body. 

St m a further object of the present invention to provide a method for 
efficiently detecting landmarks on a time«-depandent 3 dimensional image of 
the surface of parts of a body. 

It is a further object of the present invention to provide a method for 
10 efficiently creating a musculo-akelataf model that complies with bio-machanical 
restraints. 

It is a further object of the present invention to provide a method for 
efficiently creating a musculoskeletal model that uses less potentially harmful 
techniques for a patient. 
15 The above objective(s) is (are) acxxmpMshed by a method and device 

according to the present invention. 

The invention relates to a computer based method for obtaining a 
musculoskeletal modal of at Imsz part of the body of a creature, the method 
comprising ilm steps of providing a series of time-dependent .optica! 3 
20 dimensional images of the surface of said at least part of the body of a 
creature, detecting anatomical surface information based an the topography of 
the surface in said series of time-dependent optical 3 dimensional images of 
the surface of said at least part of she body erf a creature and reconstructing 
internal structures based on said detected landmarks. The anatomical surface 
25 information may be anatomical landmarks and/or the. shape of the surface. 
Providing a series of iime^dependent optical 3 dimensional images of the 
surface of said body of creature may comprise a norvcontact and~non-tftvasiVe 
scanning method. For example, It may comprise an optical method such as 
structured light projection. Providing a series of time-dependent optical 3 
30 dimensional images of the surface of said body of creature may furthermore 
comprise using raster fine (riangulation. Providing a series of time-dependent 
_ optical 3 dimensional images of the surface of said body of creature may 
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comprise using stereoscopic techniques. Providing a series of tlrr*e«d<sspenideni 
optical 3 ctinriensionat images of tiro surface of said tody of creature may also 
comprise obtaining sasti 3 tfmensional images whereby each of a width, height - 
and depth of said 3 dimensional Images can have a length up to at teast 1,2m. 
5 Said providing a dynamic 3 dimensional image of the surface of said rat least 
part of lh& body of a creature may also comprise using a mufti-view system. 
Detecting anatomical surface informaflon based on the topography of the 
surface in said! series of tome dependent optica! 3 dimensional images may be 
obtained by active contour modelling. The active contour modelling may foe 
10 based an optimising a finite number' of active contour points, whereby aH active 
contour points substantially being at an equal distance. Detecting anatomical 
surface information based on the topography of the surface an said series of 
time-dependent optica! 3 dimensional Image may be obtained by active shape 
modelling. Reconstruction of internal structures may comprise at 3&ast one of 
15 the group of bones, ligaments, tendons and muscles. The reconstruction may 
be topographically or topological correct. 

The invention ateo relates to a method for collecting data suitable for 
diagnostics of disorders In bodies of creatures, comprising building a computer 
based rnuscula-skeieial model obtained! according to a method as described 
' 2D above. 

The invention also relates io a system for obtaining a musculoskeletal 
model of at least part of b the body of a creature, the system composing 
means for providing a series of time-dependent optical 3 dimensional images ■ 
of the surface of said at least pare of the body of a cmatum, means for 

25 detecting landmarks on said series of lime dependent optical 3 'dimensional 
images of the surface of said ax (east pari of a the body of a creature and 
means for reconstructing interna! structures based on said detected 
landmarks. The m©sns for providing a series of time dependent optical 3 
dimensional images of the surface of said at least part of the body of a 

30 creature may comprise means for obtaining optical 3 dimensional images of 
the surface of said at feasf pari! of the body of a creature. 

The invention furthermore relates to a computer based method for 
detecting- and/or extracting anatomical features on surface measurements, ' 
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said method comprising, the steps of providing a series of time-dependent 
' images of the surface of foody parts of a creature and using invariant feature 
analysis to determine anatomical landmarks and shapes, wherein said 
invariant feature analysis comprises fulfilling predetermined conditions 
5 describing topographic, characterises, of the surface off the body parts of a 
creature and fiM®m predetermined conditions describing topographic, 
topologic and/or volumetric characteristics of the interior of the body parte of a 
creature. 'The topographic characteristics of the surface of the body parts of a 
creature may be at least on© of curvature and symmetry of surface parts of the 

10 body pans of a creature and wherein the topographic, topologic and/or 
volumetric characteristics of the interior of the body parts of a creator© may be 
at feast one of the relative position, bending!, torsion, equidistance and 
dynamical properties of interior parts of the body parts of a creature. The 
topographic characteristics of the surface of the body parts of a creature may 

15 be" all of curvature and symmetry of surface parts of the body parts of a 
creature and the topographic, topologic and/or volumetric characteristics of the 
interior of the body parts, of a creature may be all of the relative position, 
bending, torsion, equidistance and dynamical properties of interior parte of the 
body parts of a creature. The predetermined conditions describing topographic 

20 characteristics of the surface of the body parts of a creature and the 
predetermined conditions describing topographic, topologic and volumetric 
characteristics of the interior of trie body parts of a creature may be determined 
by biomechanieai constraints. 

The invention also relates to a computer program product for executing 

25 the method' as described above, The invention furthermore relates to a 
machine-readable data storage device storing that computer product- The 
invention also relates to transmission of that computer program product over a 
local or wide area telecommunications network. 

The invention -also relates to a computer based method for constructing 

30 a biomechanics! model of a musculoskeletal structure of at. least part of a 
body of a creature, the method comprising obtaining ime-dependent 
anatomical surface information., determining from said tome-dependent 
anatomical surface information a set of boundary conditions -for a 
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biomechanics model of a musculoskeletal structure and fitting a bio- 
mechanical model of e muscuio-sketelal structure according to sa*d set of 
boundary conditions. The anatomical surface information may comprise both 
landmarks as surface shapes. The method furthermore may comprise initially 
5 scaling an calibrating said biamechanicaf model of a musculoskeletal structure 
taa^ed on anatomical surface information obtained -for said at least pad o? a 
body of a creature in a predefined position. The method also may comprise, 
after fitting a btomechanical model of a muscuto-skeletai structure, chocking 
the plausibility of said biomechanics^ model of a muskuio-skeietai structure 
10 with respect to biomechanical constraints. The method furthermore may 
comprise after fitting a foeomedhanical mode! of a musculoskeletal structure 
according So said set of boundary conditions, dynamically adjusting and 
refining said biomechanical model of a musculoskeletal slruclure by 
repeatedly obtaining new time-dependent anatomical surface information, 
15 determining the mew boundary conditions for a biomechanical modal of a 
musculoskeletal structure leased on said new tfme-depenctent anatomical 
surface shape information and adjusting saio biomechanical mods! of a 
musculoskeletal structure according to said s©t of new boundary conditions. 
The Invention also relates to a computer program product for executing 
20 the- method as described above. The invention also relates to a machine- 
readable data storage device storing that computer product. The invention also 
relates to transmission of that computer program product over ja local or wide 
area telecommunications network. 

The invention furthermore relates to a computer based method of 
25 extended modelling of kinematics, kinetics arid dynamics of the musculo- 
skeletal system of a moving body comprising extraction of relevant parameters 
from the bio-mechanical modes as described above. 

It is an advantage of the invention that with the equipment it is possible 
to reconstruct kinematics, kinetics and dynamics of the musculoskeletal! 
30 system of the" body, as well as to indicate and quantity pathological changes or 
abnormalities. 

It is furthermore an advantage of the invention that no preparation time 
and only a short recording and analysing time are heeded. 
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St is an advantage of the invention that it enables tie full time-dependent 
three-dimensional measurement and functions] analysis of the human body in 
a contact-free, noninvasive way, without the use of potentially harmful 
radiation. It is furthermore an advantage of the* present invention that it aliows 
5 the use of HmeHdependerrt thrae-dfmensional images of the surface of part of a 
body for construction of a rnusculo-ftkeletal model. 

It is an advantage of the present invention that, for the detection of 
time-dependent landmarks on a series of time-dependent three-dimensional 
images of the surface, both detection points and characteristics of parte of the 
10 surface are used. 

It is furthermore an advantage of the present invention that the time 
evolution of landmarks and thus the characteristics of parts of the surface are 
used during construction of the musculoskeletal model. 

Although there has been constant improvement, change and evolution k 
IS of devices in this field, the present concepts are believed to represent 
substantial new and novel improvements, including departures from prior 
practices, resulting In the provision of more efficient, stable and reliable 
devices of this nature. 

The teachings of the present invention permit the design of improved 
20 methods and systems for constructing a musculoskeletal model of at least 
pari' of a body. 

These and other characteristics, features and advantages of the present 
invention will become apparent from the following detailed description, taken in 
conjunction with the accompanying drawings, which illustrate, by way of 
25 example, the principles of the invention. This description ss given for the sake 
of example only, without limiting the scope of the invention. The reference 
figures quoted below refer to the attached drawings.. 

Brief description of the drawings 

Fig. 1 is a schematic overview of the different steps of a method for 
30 constructing a musoulo-ekeletal model according to a first embodiment of the 
present invention. 
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Fly. 2 lis a detailed schemrtic overview of a preferred mode of the 
different steps of a method for constructing a muscuio-skeietesi mode! 
according to the first embodiment of the present invention. 

Fig. 3 is a schematksal illustration of the dynamic 3 dimensional 
5 measurement of part of a body according, to an embodiment of the present 
invention. 

Fig. 4 is a schematic Blustration of the triangulatfcsn technique applied to 
-pari: of a body. 

Fig. 5 is en illustration of the equipment for constructing a musculo- 
•10 -skeletal model for pari at least part of a body of a creature according to an 
embodiment of the present invention. 

Fig. 6 is an illustration of the equipment tor obtaining time dependent 
three-dimensional images of at least pari of a body of a creature according to a 
preferred embodiment of ihe present invention. 

15 F & 7 l« a schematic representation of the stereo basis for 

illumination/detection of the equipment for obtaining time dependent three- 
dimensional images of at least part of a body of a creature according to a 
preferred embodiment of the present invention. 

Fig. S is a diagram of the correlation between projection angle, height of. 
the scanning field and distance of the mid focal plane to the projector 
according to the illumination/detection In a preferred embodiment of the 
present, invention. 

Fig. 9 is a schematic illustration of a mufti-view setup for obtaining a 
musculoskeletal model of part of a moving body according to an alternative 
25 embodiment of the present invention. 

Fig. 10 is an illustration of the obtained measured surface, according to 
a method of the first embodiment of the present invention. 

Fig. 11 is an Illustration of the obtained regular grid describing the body 
surface, according Co a method of the first embodiment of the present 
30 invention. 

Fig. 12 to 14 are an illustration of the results of feature tracing of the 
surface for part of a body based on the surface reconstruction as shown in Fig. 
10 and 11. 



20 
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Fig, 15 to 16 are an illustration of the results of a reconstruction of bony 
structures based on the information obtained during the feature tracing of the 
surface as shown in Figs, i 2 to 1 4. 

Fig. 17 to rig. 19 illustrate the is an image used for analysis of musculo- 
5 skeletal parameters based on tine reconstruction of bony structures as shown 
in Figs. 15 to 16. 

Fig. 20 is an illustration of the reconstruction of bony structures for a 
shoulder of a human body according to an embodiment of the present 
invention. 

10 Fig. 21 shows a flowchart of the method used for construction of' a 

mufjcuio-skeletai modes for pans of a body according to an embodiment the 
present invention. 

The present Invention will be described with respect to particular 
15 embodiments and with reference to certain drawings but the invention is not 
limited thereto but only by the claims. The drawings described aire only 
schematic and are non-limiting, in the drawings, the see of some of the 
elements may be exaggerated and not drawn on scale for illustrative purposes. 

Description of Iflinrtralfv* entoocEmMrts" 

20 Furthermore, the terms- first, second, third and the like in the description 

and in the claims, am used for distinguishing between similar elements and not 
necessarily for describing a sequential or chronological order, it is- to he 
■ understood that the terms so used are interchangeable under appropriate 
circumstances and that the embodiments of the invention described herein are 

25 capable of operation in other sequences than described or illustrated herein. 

Moreover, the terms- top, bottom, over, under and the like in the 
description and the claims are used for descriptive purposes and not 
necessarily for describing relative positions, ft is to be understood that the 
terms so used are Interchangeable under appropriate circumstances and that 

30 the embodiments of the invention described herein are capable of operation in 
other orientations than described or illustrated herein. 
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It is to be noticed that th© term "comprising", used in the claims, should 
not be interpreted as toeing restricted to th® means listed thereafter; it does not 
exclude other elements steps. . Thus, the scope of the expression "a device 
comprising means A and 8" should not be limited to devices consisting only of 
components A and 8. it means that with respect to the present invention, the 
only relevant components of the device are A and B. 

Similarly, ft is to be noticed that the term "coupled-, also used in the 
claims, should not be interpreted as being restricted to direct connections only. 
Thus, the scope of th© expression *a device A coupled to a device B" should 
not be limited to devices or systems wherein an cutout of device A" Is directly 
connected to an input of device 0. it means thai there exists a path between 
an output- of A and an input of B which may foe a path including other devices 



or means. 
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to a first embodiment of the present invention, the invention relates to a 
method and system for obtaining a model of the internal structure of a body, 
e.g. of an animal or human. The model of the internal structure mav be a 
mc-scufo-skateta! mode! for a part or parts of a body. The body mav be the 
body of a living or dead creature. The body may be the body of an animal 
having a skeleton and a muscle structure. The body may be the body of a 
mammal. The body thus also may be a human body. The body also mav be an 
inanimate object such as a sculpture of a mammal orr a plaster cast of a body 
whereby the sculpture of piaster cast has surface contours raormrtfing an 
underlying musculoskeletal system, fn Fig. 1 a schematic overview of the 
different steps of the invention are Illustrated The method comprises the steps 
of : 

" srabtHns ima ® ss <* movement® of part or parts of a body as a function " 
of time and in a three. dimensional representation, thus producing a dynamic 
three-dimensional time-dependent surface measurement 

' Ci@tedim of dinteal ^ «^ar* Points, areas, curves and/of shapes on 
the reconstructed ^-dimensional time-dependent surface in an automatic 
way, lb. without the use of synthetic markers attached to the surface of the 
body 
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. construction of a time-dependent three-dimensional dynamical 
musculoskeletal model to analyse and visualise clinically relevant, internal 
structures such as e.g. bones, ligaments, tendons, muscles and to analyse 
and visualise body movements, constraints, dynamics, kinematics and kinetics. 
5 The first step comprises grabbing images f movements of part: or parts 

of a body as a taction of time and in three dimensions, thus producing a 
dynamic three-dimensional time-dependent surface measurement. The Image 
obtained is an image of the surface of a body. It is a specific advantage of the 
invention that this grabbing of movement images is performed in a contact-free 

1 0 way that, is furthermore quick, relatively inexpensive and without risk for health. 
The imaging technique preferably is an opticas technique. The technique for 
imaging used may be a stereoscopic technique. The measured surface is then 
reconstructed mathematically, e.g. by fitting a dense point cloud, which is 
created based on trianguiation algorithms. Other techniques of obtaining three- 

15 dimensional surface Information- may be used, e.g. using structured lighting. 

Sn a second step an Invariant shape analysis extracts information 
consisting of points, curves, objects.areas and/or shapes that are anatomically 
relevant, e.g. the position of anatomical landmarks such as the sacrum point. 
Any suitable method of image analysis and pattern recognition may be used to 

20 extract relevant landmarks. In on© preferred technique a specific 
(mathematical) "cost" is defined in such a way that a minimisation of the cost 
leaf is to an optimal recognition of a wall defined anatomical feature, e.g. the 
shape of the spin®. Starting from an initial estimation, a point, area, contour, 
object or shape is moved fteratively over the image until the cost is minimised 

25 and appropriate properties are achieved. The cost of a point, contour, are, 
object or shape comprises two parts: an external cost and an internal cost. The 
external cost guides the point, contour, area, object or shape to a minimal cost 
position on the surface. The Internal cost describes the internal behaviour of 
• ' • the point, contour, area, object or shape itself, e.g. to avoid results thai are 

30 impossible from a biomechanical point of view. That is the internal cost 
represents a constraint on the allowable values. This technique is used to 
deled: of clinically relevant points, . areas, curves and shapes on the 
reconstructed three-dimensional time-dependent surface. 
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The third step reconstructs an interna! structure mode!, such as bony 
structures, soft tissue, ligaments and tendons in three dimensions as a 
funct on of time using rrnjsculo-skeletal models of the body. This reconstruction 
is based on the three-dimensional time-dependent information of the body 
5 surfec© itself and its features (e.g. positions of anatomical landmarks), 
depending on the part of the body that is scanned. Clinically relevant kinetic, 
kinematic and dynamic parameters am then extracted from the data. These 
modals allow diagnostic measurements of e.g. the shape of the spine, the leg 
axis foot disorders. 

10 The method and system of the invention will he described in more detail 

for a preferred embodiment. The preferred embodiment of this method 
according to the first embodiment of the present invention comprises the 
different steps, as shown in Fig. 2, for constructing a blemeehanica! modes of a 
musculoskeletal structure of at least part of a body of a creature. The 

15 preferred imaging technique used during the first step - i.e. the grabbing of 
images of movements of the body in three dimensions as a function of time - is 
structured Sight projection combined with raster line trianguiation. The invention 
• enables the recording of a part of a body surface in three dimensions as a 
function of time by projecting raster lines (a set or parallel stripes) on the body 

20 surface and by capturing these lines under a known and fixed angle with a 
camera, as illustrated in Fig. 3. In other words, a fight source projects a raster, 
i.e. a set of parallel stripes, on the body surface. The raster can be a pattern of 
thick and thin lines. When falling onto a surface to be imaged, the stripes 
generate a deformed pattern on the object as shown Irs Fig. 4, which is 

25 recorded by a camera, defining a fixed angle with the projector. It is not 
necessary to know the fixed angle nor the distance to the object to be Imaged 
if Ihe system is calibrated at the required distance and at the fixed angle. The 
camera may be at a known distance from the tight source, e.g. by using the 
intersection of two lights beams at a specific distance. Fig. 5 to Fig. 7 illustrate 

30 the illumination system used. The camera used cam be any type of camera, but 
tyoically is a digital video camera. The principle thus is* to continuously project 
a set of parallel stripes ("raster Sines") on the body and with the video camera 
record the deformed pattern. The deformed pattern that enables &. 

._.-™.,.,»,„,.„,e,,, w ,*i^ wm ,!>(IS!«^ 
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reconstruction of the surface is created from the (known or- calibrated) angle 
anc (known or calibrated) distance between the projecting and the recording 
equipment. The light source projecting the raster lines on the body may be a 
standard white fight slide-projector with a special constructed raster-line slid©, 
5 consisting of lines only.. The diverging angle of the projected image should 
preferably be low. The advantage of using this light source is that it is riot very 
expansive, as standard slide projectors may b© used. The disadvantage of 
white light projection is the relative small focus field, which typically is m the 
range of 20cm to 50cm, as well as the sensitivity to fafee light. An alternative 

1 0 approach is to use laser projectors that are able to split a laser beam in a set 
of parallel lines, e.g.. by using difractive optics. Different laser projectors can 
be used. An example is a 685 nm, low-energy - i.e. 50 mW - laser sources 
with diffiracfive optics that split. th@ beam in e.g. 65 - 130 stripes. Where lasers 
are used protection for the eyes of the person or animal should be provided. A 

15 band-pass filter can be used on the video camera to enhance the stripes. The 
main advantage off the us© of laser projectors is the extended focus 'field, i.e. 
between 75cm and 150 cm, as well as the reduced weight and volume of the 
equipment itself. The disadvantage is a slightly more expensive system and a 
slightly mar® difficult method for 'sorting out the projected stripes. In case of 

20 white light projection, the system is able to capture anatomical surface data 
within a recording range of, for example, 1,2 x i,2x 0,4 m, due to the small 
focus area; in case of laser light projection, this range is at least a cube of 1,2 
x 1 ,2x 1 ,5 m. The accuracy of the scanned surface is high in both the laser and 
the white light case, i.e. a resolution smaller than 1 mm is obtained, and this is 

25 sufficient to map and preserve the features of the body to the surface, allowing 
detecting of surface properties and anatomical features as performed in step 2 
of the method. A practical range of a low-energy time-dependent system (laser 
or white light) is 50 - 250 cm. Longer distances require more energy to 
illuminate the raster lines 'sufficiently. Shorter distances may be used but these 

30 restrict the focus field (short distance - small focus field). A typical projection 
angle (between projection path and the image path to the camera) normally 
lies oetween 20° and 30*. A smaller angle than 20° may create some 
unwanted results due to the fish-eye effects (deformation of lines), and a 
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bigger angle may restrict She scan field unnecessary {long focal length - small 
field of view). For both the laser and the white light systems, a typical projector 
ang : .e can e.g. be 22.5 degress, 'The height of the resulting scan field depends 
on where to define the focal mid-plane (plane where the system is optimal 
focused). For the example given, i.e. a projection angle of 22.5°, the height of 
the resulting scan field can be calculated as h = 2d tan( 22.5° J 2 ), where d is 
the distance from the projector to the foes! mid plane. This is also -shown in 
Fig. 8. Table 1 gives an overview of the corresponding height of the resulting 
scan field for different distances from the projector to the focal mid plane. 



Distance from the projector 
to fte focai nm plane 


Height of the resulting 
scan field 


50 cm 




100 cm 


39.8 cm 


200 cm 


78/8 cm 


250 cm 


99.5 cm 


Tab 


e 1. 



The width of the scan field is typically about 75% •-• 100% of sis height; due to 
the slightly rectangular shape of a slide (white Sight) and the refractor optics 

15 (laser). The depth field depends on the focal length, aperture, and whether 
white Sight or laser is used. The distance between the transmitting equipment 
(laser or' white light projector) and the recording/receiving equipment (video 
camera) is defined by the distance to the scan field and the angle between 
transmitter and receiver, As discussed above, this angle can be determined 'by 

20 trial-and-error; experience shows that an angle between 20° and 45" is 
acceptable. A toe flat angle (<20*) makes it difficult to calculate the depth 
values from the deformed pattern, as the degree of deformation will be tow 
with a small angle; thus the robustness and stability of the reconstruction may 
suffer (small error?; in the stripe detection are magnifying themselves due to 

25 small deformation •■'tolerance''' of the pattern). « On the other hand, a too big 
angle makes the focai plane of the video camera "loan" too much, and it is 
difficult to get the whose body focused at one time (as the focal piano of the 
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camera is tilted with the same degree as the camera is tilted to the projector). 
The use of mirrors for redirecting the field of view may open th© possibility for 
using bigger angles, on the costs of mechanical stability and with the loss of 
illumination intensity. During Imaging, either a one-sided surface, for example 
5 for in human the beck or ifie face, or a multi-sided surface, for example for legs 
or a torso, can be Imaged When the three-dimensional tlme-dependent 
surface needs to be reconstructed from more than one side - La. typically when 
the reconstruction: needs to be performed over an angle of moire than 1 00* ~ r 
e.g in case of the analysts of the upper leg, a mufti-view system can be used 
10 as shown in Fig. 8. The working distance of the system typically is in the range 
of ■*-/- 2 m. 

Th& measured surface is then reconstructed mathematically, e.g. by 
fitting a dense point cloud, which is created based on tilangulafion algorithms* 
thus obtaining a sequence of single static surfaces. Based on triangiJfaiion 
15 algorithms spatial oo-ordinates of all raster points are calculated for $ach 
frame, resulting in m dense point cfoud of randorniy distributed points, 
indicating the raster lines and describing the measured surface, as illustrated 
on Fig. 10. Using one-dirnsnsScnal linear interpolation, i.e. for example bMinear 
interpolation, the ranmrriy distributed data points are transformed to a regular 
20 grid, describing the body surface, -as shown in Fig, 1 1 . in other words, a frame 
grabber projects the digitised image on the computer screen, and the raster 
lines of each frams are traced and reconslructed mathematically by light 
intensity peak detection and line sequence analysis respectively. Artefwste and 
blur can be removed before this analysis. This technique is able to grab a 
25 suiface in a very short time interval <<1/100 sec) with a high resolution (<1 
mm). No special designed computer hardware is needed for the equipment. 
?\e standard commercially available personal computers or laptops can be 
used. The computer should preferably fulfil some minimum configuration 
requirements; especially a relative" large amount, of RAM is preferred, as the 
30 digital interface io the video camera fe directly connected with the RAM (the 
image sequence Is read into RAM and "flushed" io the hard disk when tie 
RAW! is full). Any suitabte operating system may be chosen of which Windows 
XP® supplied by Microsoft Cap.- is only one option. The operating system 
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should preferably be cor^patibJe with the digital camera interface, I. a in 
principle any operating system thai 5s compatible with the digits .camera 
interface can be used. AdeStiorially, a projector, which can provide white light 
or laser light and a fast digital video camera is preferred (e.g. IEEE 1394 
5 FireWire or USB2). The scanning frequency (frames per second) depends on 
the hardware Interface, Le. the faster the system can transmit the video 
signals, the higher the frequency the equipment is able to grab, as well as on 
the selected image size, if e.g, a standard personal computer is used, such as 
a Pentium 4 having 1 GB RAM memory, 80 GB hard disk, operating with 
10 operating system Windows XP arid furthermore having an Open GL graphical 
card and an IEEE 1394 FireWire interface, the grabbing frequencies vary 
between 15 Hz for an image size of 12813x1024 pixels and 40 Hz for an image 
size of 800x800 pixels. The pixel depth typically is of the order of 10-bit. Soma 
examples of imaging body parts of human beings sire given : 
15 *> Example 1 : spine. 

For the analysis of the spine, the back surface of a human body m 
captured in 4D by projecting and capturing a set of horizontal raster lines 
on the back surface, as is illustrated in Fig. 4 t by calculating the spatial 
coordinates of the fees for each frame as illustrated in Rg. 10, and by 
20 _ reconstructing the. surfeos through a regular grid as illustrated in Fig. 11 
• Example 2: shoulder. 

For the analysis of the shoulder complex, the bade and arm surfaces are 
saptured as mentioned in example 1: by projecting horizontal msier lines 
■from behind. Additional, the upper surface of both shoulders is captured 
25 oy projecting a set of vertical raster lines from above and by capturing the 

surface with 2 additional video cameras with the equipment shown in Fig. 
3, by emulating the spatial coordinates of the lines for each frame, end 
oy reconstructing the surface through a regular grid. 
• Example 3: pelvis. 
30 : -or the analysis of the pelvis, the buttocks and lower back surface is 

tlrne-dependently cwpiisred in a aeries off three-dimensional 'Images by 
projecting and capturing a set of horizontal raster lines on the surface, by 
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calculating the spatial coordinates of the lines for eadh frame, and by 
reconstructing the 1 surface through a regular grid. 
Thus, in a preferred! mode of the method according to the present 
embodiment, stmctured light projection combined with raster line trianguiation 
5 is used to record {a p'ari of) the body surface as a function of time sn three 
dimensions; active shapes and anatomical models trace and reconstruct the 
musculoskeletal system of the scanned body parts tiroiendependently and 
three-dsrnensionaliy. 

Alternative methods of obtaining surface information are included within 
10 the 3cope of the present invention. An example, Is obtaining 3D surface 
information by means of a stereoscopic armngemef)^ Lb, using two cam&ms to 
■ obtain two slightly different images of the same scene and then analysing the 
two images to obtain a 3D representation of the object viewed. 

In a second step an invariant shape analysis extracts information 
15 consisting of points, areas, curves, objects and/or shapes that are anatomically 
relevant, e.g. the position of anatomical landmarks such as the sacrum point. 
Any suitable method of pattern recognition or image analysis can be used. As 
one example a specific (inathematicat) "cost" may be defined in such a way 
that 3 minimisation of the cost leads to an optimal recognition of a well defined 
20 anal arnica! feature, e.g. the shape of the spine. Starting from an initial 
estimation, the point contour, object or shape is moved iieratively over the 
image until the coat is minimized and appropriate properties are achieved. The 
cos\ of a point, contour, object or shape comprises two parts: an externa? cost 
and an internal cost. The external cost guides the point, contour, object or 
25 shaoe to a minima) cost position on the surface. The Interna! cost describes 
the internal behaviour of the point contour, object or shape itself, e.g- to avoid 
results that are impossible from a biomechanicat point of view. This technique 
may be described as a constrained cost optimisation technique. This technique 
is used to detect of clinically relevant paints, curves and shapes on the 
30 reconstructed Hm&<Kmensianai time-dependant surface. The second step fe 
illustrated in Pigs. 12 to 14. 

The second slap of the invention analyses the reconstructed surface, 
white taking into account the specific properties that characterise different 
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parts, of the human body, in order to detect and to follow clinically relevant 
features on the reconstructed fee-dependent three-dimensional surface in an 
automatic way (i.e. without markers). 

For each body (part) a sat of anatomical features is defined, which is 
5 necessary and sufficient for the construction of a time-dependent three- 
dimensional personalised musculoskeletal model of the body (pari:). One 
meihod which is suitable is to extract the relevant features by defining a 
specific (mathematical) iS oost" in such a way that a minimisation of the cost 
leads to an optima? recognition of Bn anatomical feature, e.g. the shape of the 

10 spine." Starting from an initial estimation, the poi'nt(&) r curved), objects) and/or 
shape(s) related to a (part of) a body are moved itena&vely over the image until 
She cost is minimized and appropriate properties are achieved. 

The cost of an active contour or shape may comprise two- parts: an 
external co^l and an- internal cost. The external cost guides the point(s), 

15 contour(s) t object(a) and/or shape(s) to a minimal cost position on the surface 
rtsetf, based on surface properties such as curvature. The internal cost 
describes the \rmrrm\ behaviour of the point(s), coritourfs), ofojeot(s) and/or 
shape(s) itself, also In relation to each other; e.g. to avoid results thai are 
impossible from a biomechanics! point of view. The combined technique may 

20 be described as a constrained cost optimisation technique. This technique is 
used to detect- anatomical features (points, curves and shapes) on the 
reconstructed 40 surface, where- each feature is characterised by a unique 
weighted average of the internal and externa! cost terms that are defined 

foSiQW. 

25 External cost terms 
* Curvature 

A first important entity is curvature, which is dependent on local image 
properties. The back surface for exam pie has a high concavity at the position 
of the vertebra? column, especially m the lumbar and sacral regions, in each 
30 point two perpendicular directions can be found where curvatures are extreme: 
the principal curvatures ki and ?c 2 . Based on these principal curvatures two 
other curvatures can be defined, namely the Gaussian curvature K » k^k 2 and 
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the mean curvature H = « (ki + Ka). The size and sign of these curvatures 
(eg. sign of Gaussian curvature) enables the recognition of anatomical 
features" (e.g. the distinction of elliptic areas and hyperbolic areas on the 
surface). 
5 « Symmetry 

For normal, healthy people the medial sagittal plane is a symmetry plane; this 
symmetry is manifested at the surface level, and ran be used for the 
recognition of anatomical features that lie in this plane. Symmetry is applied by 
locating zones of minimal asymmetry", in each point P of the surface, the 

10 asymmetry function is defined by making a transverse! cross section and by 
compering the curvatures at points left and right of P. Based on the principal 
curvatures ki and *c 2 the curvature in a arbitrary direction a can be calculated 
as. K-{a), with a the angle in between the considered direction and the principal 
direction corresponding with the curvature kl At the same distance from P two 

15 points P M and P r :^t are defined at the left and at the right side of R, <p^ en 
& , are the ancles in between the respective principal directions at these 
points and a transversal axis. Each curvature at Pm making an angle a with 
the transversal axis has a mirror point P lieM making an angle n-a. with the same 
axis. Curvatures left and right with corresponding directions are equal in case 

20 of perfect symmetry, and are calculated as follows: 
tciqfi (ct) = tc t cos" ( a ■ - 9isfi )•+• *i sw ? " { a ~ ) 

K iS M («) = *i «« = (*" - « - ^ )-'■ *j sin 3 (*" ~ a ' " ^ ) 

Further the amount of asymmetry in between Putt end Pr; g « can be 
calculated as the integral a (from 0 to s) of the square of the difference in 
25 between the corresponding curvatures. This integral represents the 
contribution of one couple of points. The total -symmetry cost of P is the integral 
A of a over the entire transversal cross section, with h the width of the 
integration interval: 

A — — \ a dx 
b v 

30 Int ernal cost terms 



26-02-2004 IB -'26 

r 

* Relative position 

Cetam anatomical features (e.g. different parts of me shouider complex) have 
a certain degree of freedom in which they can move with respect to each 
other. Depending on the body past that has to be modelled, the nature and 
5 range of these degrees of freedom are defined in internal cost terms in such a 
way that only pfiausihle resufe are calculated. 

An example m the relative height of the spinal foodies on the spinal mid- 
line compared with the anatomical detectable features C7 (vertebra 
prominence) and L4 (lumbal vertebra 4). The relative positions of the spina! 

10 bodies between C7 and L4 on the spina? curve are estimated from clinical 
publications and from statistical data' collected from spina! x-rays. Also, the. 
relative position of the spinal curve from the back surface m estimated from *- 
formulas that are extracted from sets of x-ray studies. The rotation of the spinal 
foodies are also constrained by the rotation of the spine as a whole arid by the 

15 relation of the body relative to the adjoining bodies; thus no vertebrae is 
allowed to rotate 'tree", independent of its neighbours, and no vertebra is 
al owed to rotate to anatomical impossible" positions. An "obvious" cost is that 
the bones are not afiowad too close to the surface: they may not penetrate the 
skin. This constraint is implemented in all steps of the reconstruction!, allowing 

20 faSse detections to m corrected in the iteration processes. Other relative- 
position constraints include the whole range of bone position relative to each 
other (left-right scapula relative to spine and davicula, etc) and, as mentioned 
above, the degrees of freedom and movement range for the different joints. 

* Bending 

25 5r order to avoid results that are biomechanically, clinically or dynamically 
impossible, bending m added as an internal cost. Including the bending of 
curves and shapes (or bending difference with a reference value) significantly 
improves the defection of anatomical features. With s(i) the natural 
parameterisation alongside a curve for example, and 3 r (/)the 3D co-ordinate of 
30 a snake point for 2 s i < »-l , the mathematical description of the bending 
v actor difference m as follows: 

|&[ = y {\s{( + !) - 23{i) + 3? (/ - l)J - bendmgr&f] f 

— mmmm^mm mz?M 
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» Torsion 

Torsion - and the continuity of the torsion function alongside a curve or shape « 
is included as an internal cost, in order to calculate the torsion at a certain 
curve or shape points/) with 2 , the direction of the tangent lines to 

5 the .curve in the points andS(i+l) is first calculated with a forward and 

backward differentia? respectively, being ?(j~V} »J(0 -S(i-t) and 
f ( / -:- 1) = s(i + 1) - .?(/') - The torsion f(i)in the point 5(0 can be calculated, 
starting from the normal fftf-l) and »(/ + 1) and the binormai in the 

■f 

surrounding points ?{»-l)andJ<*i + J): 
10 6(/-I) = F(i-1)xff(f~l) and *(<+l)«r(/ + l)x «(/+!) 

f( . )= ^ + I)-^-l) 

* Equidistance 

During the calculations points might mount up at places with a high curvature 
or symmetry, so that the bending and torsion will be minimized at these places. 
15 To avoid this effect an Interna! cost m inducted to keep all points that describe 
a cujve and/or shape at an equal distance, 

• Dynamical properties 

Because time-dependent measurements ere performed, an extra internal cost 
can be added guiding the calculation of each new time frame by using prior 

20 knowledge on the motion.. As an example, a Kalman -filter can be used to 
predict the position of an anatomical feature in the analysed frame using the 
calculated position of the feature in the previous frame sind the measurement 
of the frame, In this way it is possible to calculate positions of anatomical 
features that in some frames cm not fa© reconstructed from the measurements 

25 only. 

As an example., detection; of anatomical relevant: landmarks is described 
for the spine, the shoulder and the pelvis of a human body : 
® Exa mpl e 1: s pine 

For the analysis of the spine, active contours and shapes are used to 
30 locate the sacrum point, the dimple points and the vertebra prominens, and 
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tc trace the line through the process! spinosi. This is; described in more 
detail by Kass et si in international Journal of computer vision 1, p 321- 
331. At first, the sacrum -point and the vertebra prominens are located as 
the beginning and the end point of the Sine through the spinous processes. 
5 7 he externa? cost is a weighted combination of the mean and Gaussian 
curvature. 

Therefore the principle curvatures (k,,*^) are calculated in each point of 
Vie back shape. The sacrum point (SP) and vertebra prominens (VP) are 

s — - 

located on the surface using a unique combination of the local maxima or 
10 minima in the mean and Gaussian curvature. The mean curvature (M) and 
ihe Gaussian curvature (K) are calculated as 
/./ = Ifa + k-j ) and k - 

The internal cost comprises the relative dynamic position of the sacrum 
point., the dimple points and the vertebra prominens with respect to each 
. 15 other. 

Secondly, an active contour Is introduced to locate the Sine through the 
spinous processes. An asymmetry -function is used as a first external cost 
to calculate the minims! asymmetry point in each horizontal cross section, 
The entire profile of minima! asymmetry Is found by connecting these 

20 symmetry points, in order to avoid results that are impossible from a 
biomechanics! point of view.. Internal costs are added. First bending and - 
torsion costs are included. These terms are related to the smoothness of 
the curve, preventing the curve from biomechanicaily impossible 
positioning. During the calculations, active contour points will mount up at 

25 places with high surface curvature and symmetry. To avoid this effect, a 

internal cost is included to keep alt active contour points at an equal 
distance. Furthermore, an internal cost is added describing the observed 
biomechanics relation between the lateral, deviation and the axial rotation 
of the vertebrae. The result is shown in Fig. 14. 

30 * Example 2: .shoulder. , 

For the analysis of the shoulder complex, active contours and shapes are 
used to define the coordinates of the elbow, the coordinates of the 

* • |P095S^2P^ebf04i;|g%::2ll 
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acromion and the Margo fytediafis on the scapula. Furthermore the 
direction of the upper arm, the t=capu!a4horax contact and the 
scapulohumeral rhythm are calculated in order to provide a set of 
anatomical features that Is necessary and sufficient to build a personalized 
5 time-dependent three-dimensional mode! of the shoulder complex. 

For iho coordinates of iba elbow the mean and Gaussian curvatures are 
external costs; the interna! cost comprises the dynamic relative portion 
with respect to the previously captured IFrame(s). For the coordinates of the 
acromion, the moan and Gaussian curvatures are external costs; the 
10 .niemai cast comprises the relative position with reject to other 

anatomical features., such as the position of the medial line (Mfargo 
MecBaBs) and the position of the- humeral head. 

The Margo Madsalis is calculated by minimising a weighted average of 
externa! costs (mean and Gaussian curvatures) and internal costs (the 

15 relative position with respect to olhw features, the eight with reaped to 
entire Image, the dynamic- relative position with respect to the previously 
captured frarne(s) t and statistical information on the its position from elbow 
measurements). For the direction of the upper ami, surface coordinates 
are defined as external cost, and the relative position with respect to other 

20 anatomical features is defined as internal cost. The scapula-thorax contact 
is based on the relative position of anatomical features (interna! cast). The 
scapulohumeral! rhythm is based on statistical Information on the position 
of the Margo Mediatis from elbow and measurements (internal cost)- 
The calculations of ins anatomical features of the shoulder complex are 

25 calculated in one single equation structure, which is necessary because 
different features are interrelated. 
13 Example 3: pelvis. 

For the analysis of the pelvis, the position of 'the dimple points is 
calculated. The external cost comprises a weighted combination of the 

30 mean and Gaussian curvature. The internal cost comprises the relative 
dynamic position of the dimple points with respect to other anatomical 
landmarks. 
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The third step comprises the reconstruction of an internal structure e.g. 
a clinically relevant internal structure (e.g. bones, ligaments* tendons, muscles) 
as a function of time In three dimensions using musculo*akeleitsd models of the 
human body, as illustrated on Flos, 15 to 16. This reconstruction is based on 
5 the time-dependent three-dimensional information of" the body surface itself 
and its features (e.g. positions of anatomical landmarks), which can be points, 
curves and/or surfaces, depending on the part of the human body that is 
scanned, Relevant kinetic and kinematic parameters are than extracted from 
the data in order to analyse and visualize body movements, constraints and 
10 dynamics. Tm models allow measurements of e.g. the shape of the j spina, the 
lag axis, foot disorders. Thanks to the plenitude of information (anatomical 
points, curves and surfaces), the time-dependent three-dimensional! model is 
pemonalised- to a much higher degree than models using only marker 
positions. The matching of the frames (to obtain dynamic measurements) 
15 takes place with the obtained mode! from the previous step, in order to make a 
maxima! use of the information from the successive images, a combination of 
mathematical and statistical methods is used, e.g. a Kalman filter, which is a 
set of mathematical equations that provides an efficient computational 
(recursive) solution of the least-squares method. As a result, skeletal 
20 movements, joint moments and muscle strengths can he analysed as a 
function of time in three dimensions as illustrated cm Figs. 17 to 19. Sn addition, 
the musculo-skefeta! model can be animated. Thanks to the fact that 
anatomical points, curves and surfaces are measured together as a function of 
tinm, the modal is able to incorporate a self-checking feature. Positions andfor 
25 movements that are impossible from a biomechanical or anatomical point of 
view can be excluded and/br adjusted, e.g. in such a way that specific {parte 
of) bones or muscles have a certain time-dependent position with respect to 
the body surface, and in such a way that movement of these parts occurs at 
speeds ihafr are plausible from a biomechanical or anatomical point of view. 
30 Other techniques cannot have this eelf-cheddng ability, as they do not dispose 
of time-dependent Information {e.g. X-rays) or curve/surf acs information (e,g, 
measuring techniques using markers). 
* Examp le, 1 : _spme> 
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The internal spine is reconstructed from the external spine using an 
anatomical formula, estimating the distance of the skin to the centre of a 
vertebra! body, as. illustrated on Figs. 15 and IS. Spinal parameters {e.g. 
lumbar iordisis angle) can be monitored as a function of time in three 
5 d intensions. A detailed deduction of the anatomical formula is described by 
Drerup at ai. in Clinical Biomechanics 9 p28-3S. 
* Example 2: shoutdgL, 

The shoulder is a complex joints and It is not possible to mode? it as a 3 
d«gree©-of-*eedom spherical joint The skeletal model, as shown in Fig. 

10 20, contains the following bones: the sternum, clavicula, scapula and 
humerus. Between the- bones, three joints have been defined: the 
sternoclavicular pint, the acromioclavicular joint and the glenohumera! 
joint These joints are modelled as three-degrees-of-freedom (OOFs) 
spherical joints. The scapuiothoracic joint is modelled in such a way that 

15 the scapula is able to move freely with respect to the thoracic wail, in order 
to enable winging. The set of measured anatomical features is necessary 
and sufficient to define ai! DOPs of the system, and to build a personalised 
time-dependent three-dsmensional model of the shoulder complex. 
» Example 3: pelvis. 

20 A personalised time-dependent three-dimensional model of the pelvis is 
reconstructed from the sower back surface and the position of the dimple 
points. 

In conclusion, the method of the above described embodiment comprises, the 
grabbing of surface images of a moving body by suitable scanning or 

25 projection equipment, e.g. optical equipment. Thereby either a one-sided 
surface (like human back, face, ...) or a multi-sided surface (legs, torso, ...} Is 
recorded. The method furthermore comprises the reconstruction and mapping 
on a regular grid of the surface,, -thus giving a sequence of single static 
surfaces. On the regular surfaces, invariant features are. calculated, like 

30 curvature and symmetry and invariant features rare used for detecting 
anatomical landmarks and shapes on each of the static surface, like vertebra 
prominence, sacrum point (rima anl), left and right dimples, acromium, 
scapula, spinal symmetry line {process! spinosi). In the next step, the 
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landmarks and shapes are dynamically verified on the sequence, thus 
producing a time dependent set of landmarks and shapes and finally, the 
biomechanics! models are adapted to the dynamic set of landmarks. This 
process is illustrated sn Fig. 21 . 
5 ' A further embodiment of the present invention relates to the method of 
performing detection of time-dependent landmarks based on time-dependent 
three dimensional images of a surface as described in the second step of the 
first embodiment. This method of detection of time dependent landmarks can 
be used in a wide variety of applications and thus is not restricted to the 

10 specific use in me first- embodiment, restricted to musculo-skeletal models. In 
other words, the method for detecting time-dependent landmarks on a surface 
based on the topography of a. surface of a part or parts of a body of a creature 
in g| ser j SiS 0 f time dependent images of that surface Is an independent 
invention. The specific features of this method are identical to the features 

1 5 described in step two of the first embodiment of the present invention. Defining 
the costs as described in step two of the first embodiment allows to obtain an 
efficient way of detecting time dependent landmarks. St is an advantage of 'the 
current embodiment that it uses not only point information but also shape 
information of the surface. This also allows to detect anatomical landmarks in a 

20 mof-e efficient way, compared to methods for detecting landmarks that are 

known from the prior art. 

Another embodiment of the current invention relates to the actual 
building of a time-dependent three-dimenstonal rnusculo-skeietal model, as 
described in the- third step of the first embodiment. This method is not 

25 restricted to a specific imaging technique or' to a specific method of obtaining 
the anatomical surface information but is an independent invention. The 
features of the method for building a time-dependent three-dimensional 
musculoskeletal model are the same as those described in the third step of 
the first embodiment of the present invention, it is a specific advantage of the • 

30 present embodiment that it uses the obtained anatomical surface information 
to create a set of boundary conditions for a biomechanics model and that the 
mechanical model is build according to those boundary conditions. The 
method of the current embodiment furthermore Includes ;th© checking whether 
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the modes fulfils these boundary conditions during further dynamical 
development of the- bodies, i.e. during further movement and thus during 
dynamica? changes of the mod©! for musculoskeletal structure. The latter 
allocs to biomachanically correct modelling of part or parts of the foody of a 
5 creature. 

In accordance with further embodiments, the present invention includes 
a computer prograrn product which provides the functionality of any of the 
methods according to the present invention when executed on a computing 
device. Further, the present invention includes a data carrier such as a CD- 

10 ROM or a diskette which store* the computer product in a machine-readable 
form and which executes at least one of the methods of the Invention when 
executed on a computing device. Nowadays, such software lis often offered on 
the Internet, hence the present invention includes transmitting the printing 
computer product according! to the present invention ov& a local or wide area 

15 network. 

Other arrangements for accomplishing the objectives of the method and 
system embodying the invention w?R be obvious for those skilled in the art, ft is 
to he understood that although preferred embodiments, specific constructions 
ami configurations, as well as materials, have been discussed herein for 
20 devices according to the present invention, various changes or modifications in 
form and detail may be made without departing from the scope and spirit of 
thta invention, 
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Claims 

1. . A computer based method for obtaining a rnusculo-sksfeftai model of at 
least part of the body a£ a oraalurB, the method comprising the steps of 

5 - providing a series of time^pendent optical! 3 dimensional images of 

the surface of said af least part off the body of m creature 
- detecting anatomical surface information. based on the topography of 
the surface in said series of time dependent apical 3 cfimensional 
image of the surface of said at least part of the body of a creature - 
10 - reconstructing internal structures based on said detected landmarks.. 

2. A method according to the previous claim, wherein said providing a 
aeries of time-dependent optica) 3 dimensional images " of the surface of 
said body of a creature comprises using structured light projection. 

3. A method according to any of the previous claims, wherein said 
15 providing a series of time-dependent optical 3 dimensional images of the 

surface of said body of creature comprises using raster line triangutation. 

4. t A method according to any of the previous claims, wherein said 
providing a series of fires-dependent optical 3 dimensional images of the 
surface of safcJ body of creature comprises using .stereoscopic technique®, 

20 5. A method according to any of ih© previous claims, wherein said 
providing a series of time-dependent optica! 3 dimensional images of the 
surface af said body of creature comprises obtaining said 3 cSmensionaf 
images whereby eacft of a width, height and depth of sard 3 dnfmnsioratf 
Images can have- a length up to 1,2m.. 
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6. A method according to any of the previous claims, wherein said 
providing a time-dependent 3 dimensional image of the surface of said at 
least part of the body of a- creature comprises using a multi-view system. 

7. A method according to any of 'the previous claims;, wherein said 
5 detecting anatomical surface information based on the topography of the 

surface In said series of time-dependent optical 3 dimensional images 5s 
obtained by active contour modelling. 

8. A method seconding to claim 7, wherein said active contour modeling is 
based on optimising a finite number of active contour points, whereby all 

1 0 active contour points substantially being at an equal distance. 

9. A method according to any of the previous claims, wherein said 
detecting anatomical surface information based on the topography of the 
surface in said series of time-dependant optical 3 dimensional "image is 
obtained by active shape modeling. 
15 10. A method according to any of the previous claims, wherein said 
reconstruction of internal structures comprises at. feast one of the group of 
bones, ligaments, tendons and muscles. 
11. A method for collecting data suitable for diagnostics of disorders in 
creatures, comprising building a computer based musculo-skefatai model 
20 obtained according to a mashed of any of the previous claims. 

12. A system for obtaining a muscufo-skelefai mode! of at least part of a 
creature, the system comprising 

- means for providing a series of .time-dependent opticas 3 dimensional 
images of the surface of said at least part of a creature 
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- means for defecting landmarks cm said series of time dependant optical 

3 dimensional Images of the surface of said at least part of a living 
creature 

- means for reconstructing Internal structures based on said detected 
5 landmarks. 

13. A system according to claim 12, wherein said means for providing a 
series of erne-dependent optica! 3 dimensional images of the surface of 
said at least pert of the body of a creature comprises a means for obtaining 
ooiical 3 dimensional images of the surface of said at least part of a 
10 cfeature. 

14. A computer based method for' detecting and/or extracting anatomical 
features on surface measurements, said method comprising the steps of 

- providing & series of time-dependent Images of the surface of body 
pans of a creature 

- using invariant feature analysis to determine anatomical landmarks and 
shapes 

wherein said invariant feature analysis comprises fulfilling predetermined 
conditions describing topographic characteristics of the surface of the bpdy 
parts of a creature and fulfilling predetermined conditions describing 
20 (orographic, tgpotogic and/or volumetric characteristics of the interior of the 
hcdy pari® of a creature. 
15. A computer based method according to"' claim 14, wherein said 
topographic characteristics of the surface of the body parts of a creature 
are at toast one of curvature arid symmetry of surface parts of the body 
parts of a creature- and wherein said topographic topologies and/or 
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volumetric characteristics of ih© interior of the body parts of a creature are 
a? least one of toe relative position, tending, torsion, equidistance and 
dynamical properties of interior parts of the body pails of a creature, 
16. A claim according to claim 15, wherein said topographic characteristics 
5 d the surface of the body parte of a creature are curvature and symmetry 
oi surface parts of the body parte of a creature and said topographic, 
topologic and/or volumetric characteristics of the interior of the body parts 
of a creature are the relative position, bending, torsion, equidistance and 
dynamical properties of interior parts of the body parts of a creature, 

10 17. A claim according to any of claims 14 to 16 wherein said predetermined 
conditions describing topographic characteristics of the surface of the foody 
parts of a creature mid said predetermined conditions describing 
topographic topologic and volumetric characterises of toe interior of the 
body parts of a creature ere determined by biomechanics! constraints.. 

15 18. A computer program product tor executing the method as claimed in 

any of claims 14 to 17. 

19. A machine-readable data storage device storing the computer product 

of" claim 18. 

20. Transmission of- toe computer program product of claim 1 8 over a local 
20 o=." wide ares telecommunications network. 

21. A computer based method for constructing a biomechanics! mode! of a 
musculoskeletal structure of at least part of a body of a creature, - said 
rrethod comprising 

- obtaining rime-dependent anatomical surface information 
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- determining from said time dependent anatomical surface information a 
sat of. boundary c:ondffions for a biomechanics! mode] of a musculo- 
skeletal structure 

- fitting a biCHmechanicat model of a musculoskeletal structure according 
to said set of boundary conditions. 

22. A computer based method according to claim 21, wherein said 
anatomical surface information comprises both landmarks as -surface 
shapes. 

23. A computer based method according to any of claims 21 and 22, 
wherein said method furthermore comprises initially scaling an calibrating 
said foio-mechanicai mod©! of a musculoskeletal structure based on 
anatomical surface Information obtained for seed at least part of a body of a 
creature in a predefined position* 

24, A computer based method according to any of claims 21 to 23, wherein 
said method furthermore comprises, after fitting a biomechanics! modal of a 
nusculo-skeletaf structure, checking the plausibility of said biomechanics! 
node? of a muekulo-skeletal structure with respect to biomechanics! 
constraints. 

25. A computer based method according to any of claims 21 to 24, wherein 
said method furthermore comprises, after fitting a biomechanical mod©! off a 
musculoskeletal siruciuw according to said set of boundary conditions, 
dynamically" adjusting and refining said biomechanical model of a musculo- 
skeletal structure by repeatedly 

- obtaining new time-dependent anatomical surface information 
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- determining Sis new boundary conditions for a biomechanical model of 
, a muacuia-steietal structure based on said new time-dependent 

anatomical surface shape information 

- adjusting said btomechanfcal model of a musculoskeletal structure 
5 ac-cordmg to said set of new boundary conditions. 

26, A computer program product for executing tie method m claimed in 

any of claims 21 to 25. 
27\ A machine readable data storage device storing the computer product 

of claim 28, 

10 28, Transmission of the computer program product of claim 25 over a local 
or wicte area totecammunk^fcms network. 
29, A computer based method of extended modelling of kinematics, kinetics 
and dynamics of the musculo-skdtetal .system of a moving body comprising * 
extraction of relevant parameters from the bio-mechanical modal according 

15 to any of claims 21 to 25. 
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Abstract 

TIME-DEPENDENT THREE-DIMENSIONAL MUSCULO- 
SKELETAL MODELING BASED ©M DYNAMIC SURFACE 
5 .. MEASUREMENTS OF BOOSES 




Active contour models and active shape models were developed for the 
detection of the kinematics of anatomical landmarks on sequential back 

10 surface measurements. The anatomical landmarks correspond with the 
spinous processes, the dimples of ifre posterior superior iliac spines {PSiS), 
ill© margo medialis and the elbow. Back surface curvatures are used as a 
basis to guide the ACM and ABM's towards interesting landmark features en 
the back surface. Geometrical bending and torsion costs, and the main modes 

15 of variation of the landmark points are added to the models in order to avoid 
unrealistic curve shapes from a bkxnechanical point of view. Reconstruction of 
the underlying skeletal structures is performed using the surface normals as 
approximations for skeletal rotations (e.g. axial vertebrae rotations, pelvic 
torsion, etc.) and anatomical formulas to estimate skeletal dimensions. 

20 

+ Fig. 13 
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producing a three-dimensional 
time-dependent optical surface 
measurement 



detection of clinically relevant 
points, curves and shapes on the 
reconstructed three-dimensional 
time-dependent surface 

; 

i 

construction of a 
time-dependent 
three-dimensional dynamical 
musculo-skeletal model 



Fig* i 
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tfiree-dimereionai time-dependent surface measurement 

projection of structured fight on booy 
.1 1. 

raster line trianguteSon 
calculate €if three-dimensional flme-depencterit C~ 4E>) body surface 
storage oi high resolution surface streaming da$s 



autom^jsc detection of futures on the reconstructed surface sjream 

surface stream of booy ' • automatic ^mentation of body parts 

n j l 

invariant contour & shaped- definition of anatomical features fa &?_ extracted for each body psjt 
snafysfs far *r»e sxsr&ctlcjn j I. 

of anatomical features deflnitfan of externa! terms fe.qV surface properties ) and internal 
(points, cyrv«s P surfeoesft j-. tsrn'ts (e.g. biomeclianfcaL aristemlCBl or dyttairtal con^t Bints) that 
or* trie 4P body surface characterise the anatomies* features to be ! jsxtractoc! 



set 



i 

of 4D anatomical &<ta that & necessity artel sufficient' for 4D patsonsMsed model of thfc body 



$eif<on'e<±ife tirne-dependent three-dimension^ musailo-skelletal [inctfeBintj 



4D anatomical op&mai mfomratron (e*g. reference movajnenS:) 



n, 



pigrsori^fis^o 4D wsctUo-skele&s! model of body 



three-dimensions? 
fcirm* -dependent 
surface InfbrmatSorc 



three-di ?rn* n sk> t? roe-deperutent 
reconstruction of Internal structures < 
bones, iiigamsrtts, tendons, muscles, ...) 



seff-check; filten'nrj to data that an* 
plauslfcfe (ItfomeehartaJly, anatomically, 
statically, dynam really, ,* 0 
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Step 1; Single or multi- 
sided dynamical surface 
scan of She moving body: 
white light, laser 



Step 2: Enaction of 
relevant surface features; 
curvature, inflection 
points arid curves, flow 
directions. extremes, 
symmetries ... 



Step 3: Detection and 
evaluation of rctevanl 
landmarks m& shapes 
on the snrftccs: fixpointe, 
curves* shapes and 
objects 



Step 4: Ajdapuon/personaf 
isiog of iinw-dependent 
bsornech&nioai musculo- 
skeletal models to the 
landmarks, shapes and 
surfaces 




Iterative Step A; Evaluation, 
of surface features for their 
static and dynamical 
(time-spaas) plausibslty: 
continuity, smoothness, ... 



Iterative StepB: Dynamical 
time-space evaluation arid 
correction of landmarks and. 
shapes for their anatomical 
and biomechanics! 
plaustbiUy 



Iterative Step C: Dynamical 
time-space evaluation ah<J " 
correction of lbs models 
for their, anatomical and 
bjomechanical pfarcribilty 



Step S: Cafculate/cxtrwJt rj 

kinematic, kinetic and f 

dynamics data, ftoia the j 

personalised musculo- : 
skelatal models 



Step 62 Visualisation 
sod animation of 
protocols, ^'uiiS and 
statistics 



TOTAL P. 50 
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